With the spread of digital camera (imaging device), the cost of taking a picture has been largely reduced, thereby widely spreading consecutive shooting. In the consecutive shooting, for example, several pictures are taken in one second.
One of the uses of consecutive shooting is the technology of performing consecutive shooting not to fail in shooting the best scene, and selecting and storing an image of good quality by a camera user etc. For example, the consecutive shooting is performed by changing for each shooting the shooting conditions such as an amount of exposure, a focusing position, an amount of flash light, etc. during shooting of each image, and a camera user selects an appropriate image from among a plurality of obtained images.
In addition, a technology of automatically selecting an image is well known instead of the camera user etc. manually selecting an image. With these technologies, to evaluate the quality of an image, the blur of an image caused by the camera shake during shooting and an inappropriate focus adjustment is checked, and an image with less blur is selected. Especially in an image of a small depth of field, there is the technology of avoiding the undervaluation of the quality of the image due to the blur of the background outside the area by evaluating the quality of the image only in the evaluation area partially provided in the image.
As another background technology, there is the technology of separating the image of the background from the image of a mobile object in a picture of the mobile object moving in front of the static background by considering the combination of the background image and the object image. In this technology, based on the correlation of the components of the images corresponding in a plurality of time-series screens, the boundary of the screen on which the components of the images discontinuously change in the direction of time is detected for each pixel. Then, it is determined whether or not the pixel value generated based on the components of the images on the screen of at least one of the front and back screens with respect to the detected boundary belongs to at least one of the foreground and the background.
In addition, there is a known technology of generating an image which looks like a portrait picture in which a main subject looks popped out by the intentional blur of the foreground or the background. In this technology, a shot image is sectioned into a first area and a second area as another area. Then, the blur level of the second area is detected, and the blur level of the second area is enhanced in proportion to the detected level, thus performing the blur emphasizing process.
Furthermore, there is the technology of generating an image without a position shift by a movement of a camera and a blurred subject by a movement of a subject. With a device for combining a plurality of images having the same imaging range in this technology, a position shift among a plurality of images having the same imaging range is detected, and the position shift of the images is corrected. Then, after the correction of the position shift, the similarity among the respective portions in the image is evaluated, and the image synthesizing method is changed depending on the evaluation of the similarity. In this technology, using a well known method called a KLT tracker (Kanade-Lucas-Tomasi Feature Tracker) method, a feature point is extracted from the image, and traced to detect the position shift of the entire image.
As another related technology, there is the technology of detecting a position checked by analyzing the line of sight of the eyes displayed on the image of a human face. Furthermore, there is the technology of detecting the image of the face of a human as a subject from a shot image, recognizing the facial expression by the image, and calculating the level of the smile on the facial expression.
The technologies described in the following documents are also known.    Document 1:
Japanese Laid-open Patent Publication No. 2001-8087    Document 2:
Japanese Laid-open Patent Publication No. 2000-209483    Document 3:
Japanese Laid-open Patent Publication No. 2003-233817    Document 4:
International Publication Pamphlet No. WO2007/049634    Document 5:
International Publication Pamphlet No. WO2007/032156    Document 6:
Japanese Laid-open Patent Publication No. H09-251342    Document 7:
Japanese Laid-open Patent Publication No. 2005-056388    Document 8:
Japanese Laid-open Patent Publication No. 2009-253848    Document 9:
Jianbo Shi, Carlo Tomasi, “Good Features to Track”, IEEE Conference on Computer Vision and pattern Recognition (CVPR94) Seattle, Jun, 1994
The blur of the image area of the main subject without the background can be caused by the movement of the subject itself other than by the camera shake during shooting and an inappropriate focus adjustment. The blur caused by the movement of the subject itself is not always limited to mis-shooting, but can be, for example, the expression of the speedy movement of a subject which is intended by the camera user. However, it is difficult to determine based on the image whether or not the blur caused in the image area of the subject is intended by the camera user.